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ABSTRACT 

A programed sequence for teaching students to solve 
vord problems was developed using a combination of the infornf%tion 
processing 'and structural variables approaches. Students using the 
sequence pi^ceeded individually through mastery of a sequence of 
objectives. In order to evaluate the program, fourth and fifth ^ 
graders were randomly selected from classes; the r^,emai"ning st.udents 
in th^se classes served as contrqls. All students 4 were given^ the 
appropriate level of the Sta^&foTd Acheivement Test as a pret'est. 
During the 11 weeks that exp^'eisw.mental * sub jects completed the Word 
Problem Program, control subjects received regu^af mathematics 
instruction. The com-putation and applications sections of the 
Stanford Acheivement Test served as posttests* Both fourth- and 
fifth-grctde experimental groups scoped higher on their respective 
applications pogfttests than the comparable control groups. (SD) 
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Abstract 



A computer assisted instruction program to teach arithmetic word probleitis 
was designed, developed and tested in a school setting. Expe rinnental groups 
of fourth and fifth graders eich ga'ined significantly when conjpared to con- 
trol groups on national standardized t«sts of arithmetic applications. 
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SUMMATIVE EVALUATION 



Richard A '"Romany 
-University oi Pittsburgh 



^ 



Teaching elementary school students to soUyrUbmetic word prob- 
lems has been a d.ff.cuU task and has generated mu/h research (Gorman. 
1967; Suydam & R.edesel, 1969). Many mathemat^ians regard arUhmet.c 
w^rd problems as crucial .n mathematical develo/ment. so the problem 
must be cr>nfrontea (Polya. 1962). Earher reports described the des.gn, 
rationale and formative evaluation of the Wordi^roblem Program, ^com- 
puter assisted instruction program' that teachers students to solve woVd 
problems (Roman ^ Laudato. 1974; Laudato / Roman, 1975; Laudato. 1975). 
This report describes a summative evaluation study ol that program in a, 
school setting. 

^ Method 



De sign 

' The study was a pretest, treatnient, post.test design, with one experi 
niental group and one control group, jhe experimental treatment consisted 
of work on the Word Problem PrograrJ. The control.group received only ' 
that iristruction in wo'rd prob^fem's provided in the normal mathematics cur- 

fourth and fifth grade; Since the 
rent tests, the results were analyzed 



riculum. The ^tudy was replieated in 
fourth and fifth gradefs received diffe 
separately for^the two grades. 
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Materials ' ^ 

Tho Word Problem Program i^tih/es a ^om.bination oi the iniorma- 
tion processing approach oi Bobrov. (T^^m and P<iige ana Simon il'^boi with 
the structural variable approach oi Loitus and Suppe^ (Lortus, 1^73, Suppes^ 
Fletcher, Zanotti, Lorton, Searle. 1973) to create and sequence instruc- 
- tional objectives, each of which consists of problem.s of homogenous dif- 
ficulty. E^'ch problem in an objective requires the same number and same 
kind of information proces'sing steps to reach a solution. 

Once in the program, students receive a sequence of objectives 
based on their individual pe rform.ance. The program (evaluates perform- 
ance continuously and tells the students when they finish each objective- 
Work within an objective consists of a ?et of problems selected from the 
target group and practice groups that have already been mastered. The 
students cannot distinguish between target and practice problems, bvit only 
their work onf target problems contributes to the evaluation of objective per- 
formance. / 

.The literature supports the importance of students solving many 
problems when learning word problems. No method has consistently been 
found supel-^ior to a carefully sequenced set of problems (Laudato, 1 975). 
There are indications that the use of analytic steps can improve perform- * 
ance (Suydam & Riedesel, 1969). The^Word Problem Program, therefore, 
offers tbree analytic hints to each problem. Students may request those 
hints at any time. 

Afl 'calculations are earned out by the computer under the student's 
direction. ' Typical .computational commands utilize letters as variable 
n'ames, for tjxample, "A + B, " or "(C + D) /A, " or "D - C. " The decision 
to use the computational power of the computer allows the program to use 
large numbers, larger than those the student could calculate. Numbers are 
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chosen to particularly deceptive with regard to the correct operation. 
F^or example, ih single step problems, one number is always a multiple 
of the other. While this situation provides a cue for division, in the Word ^ 
Problem Program division is correct only one-fourth oi the time. In bther 
'curricula, divisibility ot the numbers reliabl> cues the division operation. 

Details about prograrrv, including th^e methods of problem genera- 
tion, problem selection, problem analyses, objective sequencing, and hint 
preparation ca^i all be found m Roman and Laudato (1<^74), amd Laudato (1975). 

Subjects 

A suburban, lower middle class, elementary school housed the com- 
puter upon which the Word Problem Program wak^implemented. The fourth 
and fifth grade teachers agreed to allow random sampling of their class- 
rooms for this, study, and accordingly, one-third of the students from each 
of four classrooms (two fourth grades, two fifth grades) were randomly 
selected for participation. In a conference before the study, the teachers 
rejected seven of the 28 randomly selected students. Reasons for rejec- 
tion included^* never finishes work, so can't afford time (three cases); 
knows h6w to'da wor>d problems, already (two cases); and behaves badly and 
is not'allowdd to use computer (two cases). The rejected students were 
replaced by^aeven other students randomly selected^f rom the same class- 



rooms. 



Procedure 



All students in fourth-tnd fifth grades took appropriate levels (Inter- 
mediate I and II, respectively) of the Stanford Achievement Tests (Madden, ^ 
Gardner, Rudman, Karlsen, & Merwin, .197 3) in January as p^rt of the dis- 
trict's normal testing procedure. The subtests on computational skills and 
mathematical applications served as pretests for the study. The computa- 
tional subtest was selected since success on word problems requires correct 
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corjputation, and students with dilferontial abihtv to corrpute i>hould per- 
forjn diiferently. Since^the Word Problorr Program requires no computa-' 
tioij. no gains we ro expected frorr the training. The n.athematKal apphca- 
tiorjs subtest consii>ts of aRproxirrate Iv ^30 word problen^.s and 10 pj;pblems 
related to graphs and charts. Gains were expected on- the subtest dihe to 
the training. , 

J In February, e^ch experimental student was instructed on-how and-^ 
whj^n to use the program and completeJT two introductory sessions. Each 
experimental subject then had a number of sessions on the program ckiring ^ 
tinie that was devoted to mathem.atics study by the control subjects. The 
nuJnber of sessions varied frorr. 4 to 47 (mean 18. 4. S. D. 10.81). Sessions 
stdpped when the teacher or student felt that learning had ceased. " r»ed- 
ba|k from the program an whic;h objectives were attempted and which were 
mastered was provided after each session to' help the teacher and student 

make their decisions. 
'Vl • . , > . * 

The study lasted eleven weeks, including one week of school vacation. 
Th.; termination of the study coincided with^re viously scheduled school- 
wide testing. When the study ended, 13 of the 28 students were still actively 
woj-king, Indeecf, several continued for five weeks after the study ended • 
ancj quit only because school closed for the summer. The other fifteen stu- 
derjts worked from three to nine weeks. 

The control students worked on theirVegular mathematics curricu- ' ■ 
lui4i when the experimental students used the computer. No special attention" 
^wa(s given to word problem study for the control group. 

I -The computation and applications subtests of the Stanford Achieve- 
njent Test^(Kelley, Madden, Gardner, ^ Rudman, 1964) were included in ' 
alspecial testing battery ^in late April. These tests were given as part of 
tl^e Learning Research and Development Center's continuous monitoring 
program mHhe school. These subtests served as the posttests for the study. 
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Results 

> 

The rav/ scores for all pretebts, computation, and applications 
v^er** analy/^ed beparate.> U) detornrmc if the expe nrxiental and control 
groups differed. The tit'h gra-'^f^ e\perimental group,scored signii icaritly 
higher than the controls on the applications pretest, F (1,43> ~ 4, 593,*p 
* . OS. No other differences were detected. Because of this difference, 
the analysis of the posttest was done as an analysis of covariance. 

» The raw scores from the tests were anajyzed by a one-way analysis 
of covaci^nce for ^acii grade level separately.^ Covariates were the two 
pretest scoces. The fourth grade experimental group scared significantly 
higher on the applications posttest, f (1,29)^ 4.398,.; <,.05, as did the 
fifth grade exper,imental group, ? - 4. 364, < . 05. No differences 

■ 

werie observed for the computation postt^t. The mean scores for all 
groups are given in Table 1. 

TaWc 1 

Applications Posttest Raw Score Mearu and Means 

' Adjusted for Pretest Scores ' ' 
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TKt Data 










^ Raw 


Adjusf^d 


' Ad|usted 


Group 






Means. 


Means 


Standi^rd Error 


Fourth' 












Control 




21 


14 86 


1539 


1 00 \ 


Experimental 




12 


19 83 • 


18 90 


1.33 . • 


Fifth" 












Control , 




30 


17 30 


19.00 


* 0.74 


Experimental 




16 


24 88 


21 69" 


, 1 00 



Note Means are jot^omparable across gracles * 

'Fourth gradp subjects took Stanford Achievement Test, Intermediate 111^^64). 
b 

Fifth grade subjects took Stanford Achievement Test, Intermediate II (1964). 



l-ooking oply. at experimental students, some measures of within- 
treatment periormance were examined. On most criteria, no differences 
between the grades were observed. Hijv,evcr, the iifth graders mastered 
significantly more objectives that the fou r-KP'gr ade r s, T i\,ZZ) - 9.7853, 
p <■ . 0 1 . The re spective me ans we re rt. 92 and 4. 92. Other prog ram van - 
ables considered included, total time in treatment, number of problem'b 
seen, time per session, and number of days worked. Means*and standard 
deviations for these measures are presented in Table 2. Since the groups 
did not differ in t^ese measures, Table 2 presents data for the- combined 
groups. 



Table 2 

Means and Standard Deviations for Internal T;eatmeni Measures 
^ Fourth and Fifth Grades Combined 

V 



Measures 




Mean 


SO 


Total time 


28 


4 625 (hours) 


83 


Total proble|T«/ 


28 


1 77 79 


80 10 


Time/problem 


28 


1.57 (minute^) 


31 


Days worked 


28 


1557 


8 89 


Time/day yvorked 


28 


18 72 (minutes) 


3 60 


Objectives mastered 


28 


804 


5 70 



m 

Discussion 

Students spending approximatel> four and a half fciours over a period 
of two and a half months working on the Word Problem Program increased 
their knowledge of arithmetic applications, as measured b> a national stan- 
dardized test, approximately five months more than their control group 
classmates. The gain^is both significant and important since it oc*curs in 
area that has traditionally been difficult to teach and was achieved with 
a small time investment. 

The fact that no gain occurred on computational skills was expected 
since the Word Problem Program provides no computational practice. The 
gain in the application posttest reflects greater problem solving skills, as 
opposed to greater computational skUlS. < _ 

The applications subtest of the Stanford Achievement Tests contains 
many problems that require 'an understanding of money, time, fractions, 
percentage, and units of measurement such as ounces, degrees, and gallons. 
None of these concepts are treated in the Word Problem Program. The 
finding of significant gains then takes on added significance since the results 
demonstrate a transfer effect from a carefully selected subdomain to a 
larger group»of problems. This implies that the fundamental problem solv- 
ing skills taught have generality beyond their initially limited scope. 

The fact that fifth graders master more obiective^s than fourth -grade rs 
IS not surprising, but in view of the unorthodox manner in which the objec- 
tives are defined and sequejiced, this finding serves to confirm at least the 
gross features of the design of objectives. Further analysis of the data 
should reveal intormation regarding the hierarchical assumptions made in 
the program. 

This study will be carried further, the data generated during each 
student session will be analyzed to determine if specific features designed « 



into the Word problem Program ai<.omplish their purpust*. Arnoug 
the aspects to be investigated are* the ofloc tivene ss ol hints, the 
effect of sequencing decisions, the vali«^'ity of the evaluation rule, and 
the homogene''ity ol problenis witfnn objeitives. 

The next goals for the program are to design a procedure to place 
students at their initial c ompe t(*nc e leve'l more quickly, probably based 
on initial performance measures vtithin ihe program, and to determine 
internally when the student has "stopped learning*/ a determination that 
currently requires teacher or student dec isic-)ryTnaking. VVith these improve- 
ment^» the program will operate more elficiently/ and th(r time required 
of each student may be further reduced. 
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